In some data processing methods, a Graphics Processing Unit (GPU) is used to perform volume rendering of volumetric data, for example volumetric imaging data obtained from a medical imaging scanner.
GPU volume rendering may entail making use of a GPU to accelerate standard volume rendering techniques, for example Multi-Planar Reformatting (MPR), Shaded Volume Rendering (SVR), Intensity Projection (IP) or slab rendering.
A typical GPU has a parallel architecture which allows the same computation to be run on different inputs many times, simultaneously. For example, the GPU may compute every pixel of an image at the same time, by applying the same algorithm at each pixel position.
A typical GPU has dedicated graphics memory, which may be located on a graphics card. For data to be operated on by a typical GPU, that data must be accessible in the dedicated graphics memory. Input data is uploaded to the graphics card for it to be operated on.
The GPU stores textures which may be, for example, one-dimensional (1D), two-dimensional (2D), or three-dimensional (3D) textures. Each element (in the case of a 1D texture), pixel (in the case of a 2D texture) or voxel (in the case of a 3D texture) may have a number of channels of some pre-defined bit depth. For example, a GPU may store textures for which each element, pixel or voxel has four 8-bit channels, which may be referred to as 8-bit red, green, blue, alpha (R8G8B8A8).
The term image may be used to refer to an array of pixels or voxels having a known dimensionality (for example, 1D, 2D or 3D). In general, an image has a known size and format.
A texture may be considered to be a container for one or more images. A texture may have a particular data format or layout. For example a texture may have a defined dimensionality (1D, 2D or 3D). A texture may have a defined channel layout, for example a fixed number and type of channels. A texture may have a defined size.
Characteristics of a texture may constrain the images that can be contained within the texture. For example, characteristics of the texture may determine how images are arranged within the texture, a size of images that can be contained in the texture, and/or a format of images that can be contained in the texture.
Definitions of a texture may be found at, for example, the OpenGL wiki (https://www.khronos.org/opengl/wiki/Texture), the Cuba programming guide (http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#device-memory-accesses) and the Texture Mapping chapter of the OpenGL red book (http://www.glprogramming.com/red/chapter09.html).
GPUs may make the sampling of textures fast by using caching and dedicated interpolation hardware. Typically, when sampling from a texture, the GPU is able to perform linear interpolation very quickly. Individual texture samples are cached, so that subsequent fetches are very fast.
Traditional volume rendering may involve uploading an entire volume to the GPU's dedicated memory as a volume texture.
It is becoming increasingly common to use large volumes. For example, in the case of medical imaging, increased medical scanner resolution may result in very large imaging volumes. Some volumes may be large enough that they cannot fit into the dedicated graphics memory of a GPU in their entirety.
It is also becoming common to operate multi-user enterprise systems. In multi-user enterprise systems, a computing resource may be shared between multiple users. In some circumstances, multi-user enterprise systems may have an even more limited memory budget per-user than single-user systems.
GPU rendering may need to adapt to a changing environment.
It is known to implement virtual texturing when uploading data to a GPU. Virtual texturing may reduce an amount of data that is held on a GPU to render a single frame.
FIG. 1 is a schematic diagram which is representative of a virtual texturing method in which data from a source texture 10 is stored in an atlas texture 20 on a GPU.
The source texture 10 and atlas texture 20 are each represented by a respective box in FIG. 1.
In the example of FIG. 1, the source texture 10 is a large texture. For example, the source texture 10 may be a texture that is too large to fit within the graphics memory of the GPU on which the atlas texture 20 is stored.
The source texture 10 is divided into a plurality of regions 12, which may also be referred to as blocks or bricks. In the example of FIG. 1, the regions are all of the same size and shape, for example cubic. The regions are shown as regular square boxes in FIG. 1.
The atlas texture 20 is smaller than the source texture 10. The atlas texture 20 is configured to store the same data type as the source texture 10.
The atlas texture 20 is divided into regions 22, for example cubic regions. In the example of FIG. 1, regions 22 are of the same size and shape as the regions 12 of the source texture 10.
A precomputation is performed in which processing circuitry precomputes the regions 12 of the source texture 10 that are needed in order to render a given image, which may be referred to as a frame.
In many cases, only a part of the source texture 10 may be required in order to render a given frame. In one example, only one feature represented in the source texture 10 is of interest and other features may be omitted from the rendering. In another example, some features represented in the source texture 10 are not visible from a viewpoint from which the frame is to be rendered because they are occluded by other features. Regions that are representative of the occluded features may be omitted from the rendering.
In the example of FIG. 1, a region of interest 14 is determined in the source texture 10. The region of interest 14 is indicated in FIG. 1 by an outlined box overlaying the source texture 10.
Regions 12 that at least partially fall within the region of interest 14 are selected for rendering. Such regions 12 are shaded in FIG. 1 and are given the reference numeral 16. The selected regions 16 may also be referred to as useful regions.
Only the selected regions 16 are uploaded to the GPU. The selected regions 16 are uploaded into the GPU by mapping the selected regions 16 into the atlas texture 20. Each selected region 16 of the source texture 10 is mapped to a corresponding region 24 of the atlas texture 20. Regions 24 of the atlas texture 20 onto which selected regions 16 of the source texture 10 are mapped are indicated by shading. Arrows 26, 28 show the mapping of two of the selected regions 16 of the source texture 10 to corresponding regions 24 of the atlas texture 20.
In mapping from the source texture 10 to the atlas texture 20, regions may be rearranged, for example reordered. In the example of FIG. 1, a diagonally-arranged set of selected regions 16 of the source texture 10 is mapped onto successive full lines of regions 24 of the atlas texture 20.
The processing circuitry also uploads to the GPU an index which stores the mapping of selected regions 16 of the source texture 10 to regions 24 of the atlas texture 20. For each selected region 16, the index stores a position of the respective corresponding region 24 of the atlas texture 20. The index is capable of mapping any voxel of the selected regions 16 of the source texture 10 into the atlas texture.
By mapping only the selected regions 16 to the atlas texture 20, GPU operations (for example, interpolation) may be performed by the GPU only on data from the selected regions 16. The use of an atlas texture 20 may greatly reduce the memory footprint used in rendering a frame in the case where rendering that frame requires only a subset of the complete source volume data.
In the example described above with reference to FIG. 1, it is required that the source texture 10 has the same data format as the atlas texture 20. For example, the source texture 10 and the atlas texture 20 may each comprise 8 bit data.
FIG. 2 is a schematic illustration of channel packing, which may also be called multimaps. In the example shown in FIG. 2, two grayscale source images 30, 32 (which may also be called source textures or source data sets) are packed into two channels of a further image 34 on the GPU.
Each of the source images 30, 32 is a grayscale image having only one channel of data. Each source image 30, 32 comprises 8-bit data.
Grayscale data from the first source image 30 is stored on the red channel of the further image 34. Grayscale data from the second source image 32 is stored on the green channel of the further image 34.
In FIG. 2, a first type of shading represents data from the first source image 30 and a second type of shading represents data from the second source image 32. Both types of shading are shown for the further image 34 to show that data from both source images 30, 32 is stored there.
Sampling may be performed by selecting the correct channel for the data that is desired (for example, the red channel for data from the first source image 30 and the green channel for data from the second source image 32).
Channel packing may exploit GPU texture caching behavior when a single GPU thread must access multiple textures at the same point. Textures using multiple components (for example, RGBA and/or further components, for example unrelated unnamed components) may be stored in an interleaved order. Storing components in an interleaved order may make it easy for a program or GPU hardware to read multiple components at once. Channel packing may exploit the fact that reading interleaved channels (for example, reading up to four interleaved channels) may take almost the same time as reading one channel. Reading one channel from each of four different images may take on the order of four times as long as reading one channel from one image.
In the example of FIG. 2, the data layout and type of the further image 34 is the same as that of the source images 30, 32. The source images 30, 32 and further image 34 each comprise 8-bit data. The data layout and typing are static.